Line detecting method using two rectangular windows passing through the line to be detected

ABSTRACT

In the preparatory stage, among all possible lines in two rectangular windows (12) and (14), assuming dust, a flaw or the like to be a part of a line, the most valid one is recognized as a line, and the inclination and characteristic values of the line are obtained, and then each rectangular window is deformed into a parallelogram-shaped window along this inclination. In the detecting stage, the inclination and characteristic values of the line to be detected are obtained by use of the parallelogram-shaped windows. Thus, even if image data is inferior, the detection of a line is possible, so that it is possible to detect a weld line, an edge of an object of detection, etc. with high accuracy.

FIELD OF ART

The present invention relates to a line detecting method for use inimage processing to detect the position of a rectilinear object, forexample, a weld line or an edge of an object. More particularly, thepresent invention relates to a method which enables detection with highaccuracy even in a case where the input image is not constant because ofinferior surroundings or in a case where the object of detection isstained.

BACKGROUND OF THE INVENTION

As a known method of detecting a line by use of an image processingmethod, as described above, there is a method wherein an image isconverted to binary form and a position that changes from black to whiteor from white to black is detected. Another method is known whereindifferentiation is performed by use of a matrix of pixels arranged in 3rows and 3 columns.

The method wherein a line or an edge is detected by converting the imageto binary form requires that the ambient brightness and the workcondition be fixed, since the binary image changes with a change in theambient brightness.

The method that employs a 3×3 matrix is affected by a local change ofthe brightness and hence cannot provide a clear image, so that, if thebrightness of the image changes gently, points where the brightnesschanges, that is, the periphery of the image, cannot be found. Inaddition, if the image has a stain or the like adjacent a rectilinearline to be detected, there is a strong possibility of inviting anerroneous detection. In addition, since a large number of arithmeticoperations are required, the processing speed is low.

Accordingly, these methods can be applied in a case where the inputimage is constant and clear, but not in inferior surroundings such asthose in which an articulated industrial robot performs an operation,for example, welding.

In view of the above-described problems, it is an object of the presentinvention to provide a method of detecting a line or an edge as anobject of detection with high reliability even in an operation performedin surroundings where conditions are not constant and the work isextremely stained.

SUMMARY OF THE INVENTION

The present invention provides a method of detecting the position of arectilinear object of detection from an image taken into an imageprocessing apparatus by an imaging device, wherein a preparatory stagethat comprises the steps (a) to (g) is first executed as follows:

(a) setting two parallel rectangular windows that cross a line to bedetected at their long sides;

(b) adding image density values in each of the pixel rows in thedirection of the short sides of the rectangular windows to obtainone-dimensional data comprising a row of the resulting added values foreach rectangular window;

(c) obtaining at least one feature point for each of the one-dimensionaldata, which has a characteristic value based on the size, gradient,change in gradient, etc., of the one-dimensional data;

(d) combining the characteristic values at the feature points in one ofthe one-dimensional data with the characteristic values at the featurepoints in the other one-dimensional data, and obtaining a combinationthat has the maximum value as the sum of the characteristic values;

(e) storing in memory the inclination of a line connecting the twofeature points that show the maximum value, together with the value ofthe sum;

(f) inclining each of the rectangular windows so that the short sidesthereof are parallel to the line connecting the feature points, therebyforming a parallelogram-shaped window; and

(g) executing the steps (b) to (e) in regard to the parallelogram-shapedwindows, thereby storing the inclination and the value of the sum.

To effect actual detection, the following step (h) is executed:

(h) executing the steps (b) to (e) again in regard to theparallelogram-shaped windows obtained in the preparatory stage, andcomparing the inclination and the value of the sum, thus, obtained, withthose obtained in the preparatory stage, thereby detecting the positionof the line to be detected.

According to the present invention, in the preparatory stage, among allpossible lines in two rectangular windows, assuming dust, a flaw or thelike to be a part of a line, the most valid one is recognized as a line,and the inclination and characteristic values of the line are obtained,and then each rectangular window is deformed into a parallelogram-shapedwindow along this inclination.

In the detecting stage, the inclination and characteristic values of theline to be detected are obtained by use of the parallelogram-shapedwindows. Thus, even if image data is inferior, it is possible to effecta detection which is adequately fit for practical use.

BRIEF DESCRIPTION OF THE DRAWING

FIGS. 1A to 1E illustrate one embodiment of the present invention;

FIG. 2 shows an apparatus required to carry out the present invention;

FIG. 3 shows pixels in one picture frame in the present invention; and

FIG. 4 illustrates line recognition according to the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

A block diagram of an apparatus required to carry out the presentinvention is shown in FIG. 2.

An analog video signal that is formed by a CCD camera 201 consists of525 scanning lines, in general. In the embodiment of the presentinvention, however, 481 scanning lines in the middle portion of thepicture frame are defined as effective scanning lines because noise islikely to be superposed on the peripheral edge portion of the pictureframe.

An A/D converter 202 converts the analog video signal into 8-bit digitalvalues (brightness levels of from 0 to 255, i.e., image density values),and the output of the A/D converter 202 is input to a sampling circuit203 where the effective image period of the above-described video signalfor one line (1 scan) is divided into 512. As a result, each pictureframe comprises a data matrix comprising 481 rows and 512 columns (seeFIG. 3), and this data is transferred to an image memory 205 through aDMA circuit 204.

A CPU 206 reads out the data from the image memory 205 and executesimage processing. The data for the image processing is temporarilystored either in an internal memory of the CPU 206 or in an externalworking memory (not shown). In FIG. 2, reference numeral 207 denotes aCRT display, and 208 a keyboard.

With the described apparatus employed, one embodiment of the presentinvention will be explained below in the processing sequence:

(a) The operator sets on the CRT display two parallel rectangularwindows 12 and 14 that cross a line 10 as an object of detection alongtheir long sides, as shown in FIG. 1A. The lengths of the long sides ofthe first and second rectangular windows 12 and 14 are set so that line10 lies within the set lengths, whereas the lengths of the short sidesare each set to several tens of pixels with a view to finding a goodbalance between the required reliability of the detection and the lengthof the processing time. In FIG. 1A, the number of pixels constitutingeach short side is set to 5 for facilitating understanding.

(b) After the rectangular windows 12 and 14 are set in this way, the CPUadds image density values in each of the pixel rows in the direction ofthe short sides of the rectangular windows 12 and 14 to obtain add data##EQU1## (see FIG. 1B).

Accordingly, this add data is one-dimensional data. This data istemporarily stored either in the internal memory of the CPU or in theexternal working memory (not shown).

FIG. 4(a) shows an image of a line 401 as an object of detection anddust particles or flaws 402, FIG. 4(b1) shows one-dimensional data inthe first rectangular window 12, and FIG. 4(b) shows one-dimensionaldata in the second rectangular window 14.

(c) At least one feature point of the one-dimensional data in the firstrectangular window 12 is found. In this embodiment, among theone-dimensional data, four values, that is, the largest value and thesecond, third and fourth largest values, are found. Reference symbols Ato D in FIG. 4(b1) show the four values. Similarly, four feature pointsin the second rectangular window 14 are found. Reference symbols a to din FIG. 4(b2) show the four feature points.

(d) Next, the feature points A to D in the first rectangular window 12and the feature points a to d in the second rectangular window 14 arecombined with each other, and a combination that has the maximum valueas the sum is obtained. That is, among 15 combinations, i.e., A-a, A-b,A-c, A-d, B-a, B-b, . . . , D-d, a combination that has the maximumvalue is selected.

In the example shown in FIGS. 4(b1) and 4(b2), the combination of C andb is the largest. In other words, the line that passes through thepoints C and b is recognized as the line to be detected.

As to the way of obtaining feature points, although in this embodiment asimple size comparison is made, the following methods may also beemployed depending upon the characteristics of the object of detection:

(i) For each one-dimensional data, the absolute value of a difference isobtained for every two pixels which are apart from each other by apredetermined number of pixels, and a position that takes a maximumabsolute value of the difference is defined as a feature point.

(ii)For each one-dimensional data, the absolute value of a difference isobtained for every two pixels which are apart from each other by apredetermined number of pixels, and only when the absolute value of thedifference value changes in the sequence: large→small→large, orsmall→large→small, the difference value is obtained, and a positionwhere the difference value reaches a maximum is defined as a featurepoint

Although there are other methods that utilizes a standard deviation ofone-dimensional data, description thereof is herein omitted for avoidingcomplication.

The reason why a difference is obtained every predetermined number ofpixels is as follows: When the input image is unclear, if a differencebetween a pair of adjacent pixels is obtained, the difference isinadequately small. Therefore, a difference between two pixels which areapart from each other by some pixels is obtained. The number of pixelswhich are skipped over is set according to the degree of clearness ofthe image.

(e) The sum of the characteristic values of the combination and theinclination thereof are stored in memory.

(f) Each of the rectangular windows is inclined along the inclination sothat the short sides are parallel to the direction of the inclination,thereby forming a parallelogram-shaped window (FIG. 1C). In this case,since the pixel matrix itself, shown in FIG. 3, cannot be deformed, dataof pixels which are to be fetched must be determined. FIG. 1D shows anexample in which the angle of inclination is 45 degrees, and FIG. 1Eshows an example in which the angle of inclination is 27 degrees.

(g) The processing steps (b) to (e) are executed in regard to theparallelogram-shaped windows, thereby storing the inclination and thevalue of the sum.

It should be noted that, although the step (g) is necessary when ahigher degree of accuracy is desired, it may be omitted for a simpleprocessing.

The foregoing is the processing in the preparatory stage, which ispreparation needed to enter into an actual operation. In particular,when the present invention is applied to an image processing for anindustrial robot, this operation is executed with a test sample at thetime of teaching the robot.

The following is a processing in the detecting stage.

(h) With regard to the parallelogram-shaped windows obtained in thepreparatory stage, the processing steps (b) to (e) are executed, and theinclination and the value of the sum of characteristic values, thusobtained, are compared with those obtained in the preparatory stage,thereby detecting the position of the line to be obtained.

By execution of the above-described method, in which image densityvalues are added in the direction of the short sides of the windows, thedensity feature of the portion 401 corresponding to the line to bedetected is emphasized more than that of the stain, dust or the like 402which has a short length in the direction of the short sides.

As has been described above, the present invention enables imageprocessing to be executed with high accuracy even in industrial sitesof, for example, a welding operation, an operation of assemblinglarge-sized machinery, etc., where the input image signal is not stabledue to a stain on the object or inferior surroundings.

In addition, even in a working area where it has heretofore beenimpossible to introduce a robot because no good image can be obtained, arobot with a visual sense capability can be introduced by use of thisimage processing. Therefore, the present invention will make a greatcontribution to saving of labor.

The present invention can be utilized for image processing, for example,in a welding robot that performs welding while monitoring the weld line.

We claim:
 1. A method of detecting a position of a rectilinear object ofdetection by image processing using an imaging device, comprising thesteps of:performing prepatory steps, with respect to a standard sample,of:(a) setting two parallel rectangular windows having long sidesextending in a long side direction and short sides extending in a shortside direction, such that said long sides cross a line to be detected;(b) adding image density values in each pixel column extending in theshort side direction of said rectangular windows to obtainone-dimensional data comprising a row of resulting add values for eachrectangular window, with each add value corresponding to one said pixelcolumn; (c) obtaining at least one feature point for each of saidone-dimensional data, each said feature point having a characteristicvalue based on one of size, gradient and change in gradient, of saidone-dimensional data; (d) combining all characteristic values at allfeature points in one of said one-dimensional data with allcharacteristic values at all feature points in the other saidone-dimensional data, and obtaining a combination that has a maximumvalue of a sum of said characteristic values; (e) storing in memory theinclination of a line connecting the two feature points that show saidmaximum value, together with the value of said sum; (f) inclining eachof said rectangular windows so that the short sides thereof are parallelto the line connecting said feature points, thereby formingparallelogram-shaped windows; (g) executing said steps (b) to (e) byusing said parallelogram-shaped windows instead of said two parallelrectangular windows, thereby storing said inclination and the value ofsaid sum; and performing the following final step where a rectilinearobject of detection is actually detected:(h) executing said step (g)again with respect to said object to be detected, and comparing theinclination and the value of the sum, thus obtained, with those obtainedin said preparatory stage, to thereby detect the position of the line tobe detected.
 2. A method of detecting the position of a rectilinearobject of detection by image processing using an imaging device,comprising the steps of:performing prepatory steps, with respect to astandard sample, of:(a) setting two parallel rectangular windows havinglong sides extending in a long side direction and short sides extendingin a short side direction, such that said long sides cross a line to bedetected; (b) adding image density values in each pixel column extendingin the short side direction of said rectangular windows to obtainone-dimensional data comprising a row of resulting add values for eachrectangular window, with each add value corresponding to one said pixelcolumn; (c) obtaining at least one feature point for each of saidone-dimensional data, each said feature point having a characteristicvalue based on one of size, gradient and change in gradient, of saidone-dimensional data; (d) combining all characteristic values at allfeature points in one of said one-dimensional data with allcharacteristic values at all feature points in the other saidone-dimensional data, and obtaining a combination that has a maximumvalue of a sum of said characteristic values; (e) storing in memory theinclination of a line connecting the two feature points that show saidmaximum value, together with the value of said sum; (f) inclining eachof said rectangular windows so that the short sides thereof are parallelto the line connecting said feature points, thereby formingparallelogram-shaped windows; and performing the following final stepwhere a rectilinear object of detection is actually detected:(g)executing said steps (b) to (e) by using said parallelogram-shapedwindows obtained in said preparatory stage, and comparing theinclination and the value of the sum, thus obtained, with those obtainedin said preparatory stage, to thereby detect the position of the line tobe detected.